Assessment of learning health system science competency in the equity and justice domain.

Seven knowledge domains were originally defined for the learning health system (LHS) scientist. To assess proficiency in each of these domains, we developed and published an assessment tool for use by emerging LHS scientists and training programs. (LHS, October 2022). In mid-2022, the AHRQ adopted an eighth LHS knowledge domain, Equity and Justice. The addition of this eighth domain emphasizes the importance and centrality of equity in the LHS and improvement science. To extend our prior LHS competency assessment, we developed a proficiency assessment for the new equity and justice domain. Content experts and trainees iteratively defined, reviewed, and edited the assessment criteria. The items were developed by trainees and experts at one LHS training center with experience conducting research focused on healthcare inequities among marginalized populations. The proficiency assessment criteria for the Equity domain apply the same four levels of mastery: "no exposure," "foundational awareness," "emerging," and "proficient" as were used for original competencies. LHS training programs can use these proficiency criteria to monitor skills among emerging scientists across the eight domains, with particular attention to equity and justice.

A national professional development program fills mentoring gaps for postdoctoral researchers.

The Postdoc Academy: Succeeding as a Postdoc was designed to build postdocs' skills in career transition, career planning, collaborative research, resilience, and self-reflection. This study examined self-reported changes in five skills as learners progressed through the course. Data were collected from participants who responded to both pre- and post-surveys and engaged with the course learning activities. Results from repeated measures multivariate analysis of variance revealed that all of the self-reported perceptions of skills improved significantly upon completion of the course. Hierarchical regressions revealed that underrepresented minority learners had greater gains in their development of skills in career planning, resilience, and self-reflection. Qualitative analysis of learners' responses to learning activities found that postdocs perceived networking and mentor support as contributing factors to their skill advancement while tensions among multiple obligations and concerns of uncertainties were significant challenges to applying those skills.

Palliative Care Challenges in Nigeria: A qualitative study of interprofessional perceptions.

CONTEXT: Palliative care awareness, education and practice vary widely across global health systems, especially throughout low- and middle-income countries such as Nigeria. Unfortunately, qualitative investigations into the context, experiences and challenges of Nigerian health care professionals providing palliative care in these settings are still underrepresented in the literature. OBJECTIVES: The aim of this study was to better understand the perceptions of palliative care providers in Nigeria. METHODS: The authors conducted an online survey of health professionals caring for patients with palliative care and end of life needs. Survey participants were recruited via convenience sampling from a palliative care training program in Lagos, Nigeria. RESULTS: 27 palliative care program participants (12 physicians, seven nurses, four pharmacists, two psychiatrists and two physiotherapists) responded to the two-question survey. 39 free text responses were collected and analyzed. A majority (33%, n = 13) of responses reported challenges associated with caring for patients with cancer. Suboptimal pain management was the second most common response type (18%, n = 7). Other significant responses included caring for patients with comorbidities (13%, n = 5), patients seeking spiritual care (8%, n = 3) and patients who were unable to afford standard treatment (5%, n = 2). Uncategorized responses (23%, n = 9) included experiences caring for patients with injuries sustained in military operations and COVID-19, among others. CONCLUSION: These results provide valuable insights into the palliative care experiences and challenges of an interdisciplinary set of health care practitioners providing palliative care in Nigeria. Further research is needed to elucidate the facilitators and barriers of delivering palliative care in similar settings.

Building the Nigerian Palliative Care Workforce: An Interdisciplinary Distance Learning Training Program.

Background: Education and capacity building in palliative care are greatly needed in Nigeria. Currently, two institutions integrate palliative care into the undergraduate medical curriculum and no post graduate training exists. A team from the University of Lagos in Nigeria and Northwestern University in the US collaborated to design, implement, and evaluate a 12-hour virtual palliative care training program for Nigerian health professionals. Objective: This study investigated the impact of the first session of the training program on healthcare professionals' knowledge, skills, attitudes, and confidence in palliative care. Methods: The Education in Palliative and End-of-Life (EPEC) curriculum and the Kenya Hospices and Palliative Care Association (KEHPCA) curriculum were used as foundations for the program and adapted for the Nigerian context. Delivered online, the training focused on goals of palliative care, whole patient assessment, communication skills, pain management, psychosocial issues, palliative care in COVID, oncology, and HIV. A mixed-methods evaluation based on Kirkpatrick's evaluation framework was used and data were gathered from surveys and focus groups. Findings: Thirty-five health professionals completed the training. The training had a positive impact on knowledge, skills, and attitudes. Confidence in providing end-of-life care increased from 27.3% to 92.9% while confidence in prescribing medication to relieve symptoms at the end of life increased from 42.9% to 92.0%. Performance on multiple-choice knowledge tests increased by 10% (p < 0.01). All participants stated that they would recommend the program to a peer while 96.4% reported the program was relevant to the Nigerian context. Qualitative analysis suggested that the training would help participants provide more holistic care for patients, communicate better, and change how they interacted with families. Topics to be addressed in future training were identified. Conclusions: This virtual training can be an important element in palliative care capacity building in Nigeria and represents a model for global health collaboration.

Development of a learning health system science competency assessment to guide training and proficiency assessment.

Introduction: Learning health systems (LHS) science is fundamentally a transdisciplinary field. To capture the breadth of the competencies of an LHS scientist, AHRQ and national experts defined a series of 42 competencies across seven domains that support success. Clinicians, researchers, and leaders who are new to the LHS field can identify and prioritize proficiency development among these domains. In addition, existing leaders and researchers will assemble teams of experts who together represent the LHS science domains. To serve LHS workforce development and proficiency assessment, the AHRQ-funded ACCELERAT K12 training program recruited domain experts and trainees to define and operationalize items to include in an LHS Competency Assessment to support emerging and existing LHS scientists in prioritizing and monitoring proficiency development. Methods: Sequential interviews with 18 experts iteratively defined skills and tasks to illustrate stage in proficiency, and its progression, for each of 42 competencies in the seven LHS expertise domains: systems science; research questions and standards of scientific evidence; research methods; informatics; ethics of research and implementation in health systems; improvement and implementation science; and engagement, leadership, and research management. An educational assessment expert and LHS scientist refined the assessment criteria at each stage to use parallel language across domains. Last, current trainees reviewed and pilot tested the assessment and the LHS Competency Assessment was further refined using their feedback. The assessment framework was informed by Bloom's revised taxonomy of educational objectives (Anderson and Krathwohl, A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives, 2001) where learning progresses from recalling, defining, understanding, and awareness at the lower levels of the taxonomy, to applying and adopting and finally to creating, designing, and critiquing at the upper levels of the taxonomy. We also developed assessment criteria that could be used for longer term assessment of direct performance. Van der Vleuten et al. (Best Pract Res Clin Obstetr Gynaecol. 2010;24(6):703-719) define longer term direct assessment methods as assessment that occurs over a period ranging from weeks to even years and involves multiple assessment methods and exposure to the learner's work over an extended period. Results: This experience report describes the content of the LHS Competency Assessment. For each domain and competency, the assessment lists examples of evidence to support expertise at each level of proficiency: no exposure; foundational (awareness/understanding); emerging (early application); and proficient (application with a high level of skill). Trainees begin with baseline standard assessment tables, where they can indicate no exposure or mark the foundational and emerging skills with which they have competence. For domains where foundational and emerging skills have been achieved, users can move on to assessment tables that list evidence of proficiency. Conclusion: The LHS Competency Assessment offers consistent, graded criteria across the seven LHS domains to guide trainees and mentors to evaluate progress from no experience to foundational knowledge, emerging proficiency, and proficiency. The assessment can also be used to design training and mentoring for those newly exposed to LHS science and for those with key expertise who wish to expand LHS expertise.

Using Evidence-Based Pedagogical Approaches to Pivot from In-Person to Online Training in a D43 Program during the COVID-19 Pandemic.

The COVID-19 pandemic caused significant disruption to medical education globally. Fogarty International Center (FIC) training programs, designed to strengthen research capacity in low- and middle-income countries (LMICs), through partnerships between United States and LMIC institutions were particularly vulnerable to COVID-19 disruptions. We adapted short-term training for our FIC HIV Patient-Centered Outcomes Research program in Tanzania to the virtual environment using synchronous, asynchronous, and blended approaches and a variety of teaching pedagogies. We evaluated the acceptability and effectiveness of the new trainings among trainees and facilitators using a mixed-methods approach. Ninety percent of trainees and Muhimbili University of Health and Allied Sciences (MUHAS) facilitators agreed that the virtual training methods used were effective. Trainees reported high levels of satisfaction with the technology, group work, and relevance to their research. More than 50% of trainees and MUHAS facilitators agreed that learning in the virtual environment was as effective as, or more effective than, traditional in-person learning. However, they desired more interaction, opportunities to ask U.S. facilitators questions, and choices about topics for online versus in-person trainings. Two-thirds of U.S. facilitators agreed that the virtual delivery method was an effective way for participants to learn the material, although they also rated interaction less favorably. Virtual training incorporating pedagogical best practices of blended learning and traditional teaching online was a feasible, acceptable, and effective way of conducting research training to junior scientists during COVID-19. Virtual learning could become an integral part of post-pandemic training with some adaptation to improve interactions.

Theory of change models deepen online learning evaluation.

Massive Open Online Courses (MOOCs) have gained traction as resources for professional development. This article presents the method that we used to evaluate a professional development MOOC for postdoctoral trainees that was created by a university consortium in the US. Most approaches to evaluating MOOCs focus on analysis of participation, outcomes from course assignments, self-reported learning outcomes, course completion and user pathways through the online content or clickstream data. Few published evaluations describe in detail how learning happens within online courses and the anticipated medium and longer term cognitive and behavioral outcomes on participants. This work aims to guide those who are designing, implementing and evaluating MOOCs through applying theories of change to focus evaluation on the process of learning. This approach can be used as a complement to traditional approaches for evaluating MOOCs. We described how we worked with the MOOC team building the content to develop a theory of change for each module (or lesson) within the MOOC and used the theory of change to guide evaluation of short and medium term participant cognitive, affective and attitudinal, and behavioral outcomes. Finally, we share lessons learned and suggestions for implementing theories of change in both the design and evaluation phases of MOOC development.